Track runtime shutdown and retain foreign cleanup - #64

Summary

Make Runtime startup, stop observation, and shutdown explicit.
Use upstream TaskTracker, dispatcher handles, and abort-on-drop guards.

• Add non-executing Runtime::prepare and fallible start_execution. Retain partial-startup rollback until cleanup ends.
• Accept stop promptly and return typed Drained / Forced shutdown outcomes. Both successful outcomes prove tracked execution and registered cleanup ended.
• Finalize foreign scheduling before the durable snapshot. Retain registered cleanup independently of abortable dispatchers.
• Report known turn faults before later provider awaits. Record failures before protected diagnostics run.
• Add Runtime::stop_status() and an owned RuntimeStopObserver. Observation retains status metadata, not the Runtime or its callbacks.
• Report aggregate pending foreign retirements without per-Runtime metric labels.

The branch now includes main 36b789d through merge commit
eae7780.
Upstream nonblocking activity admission retains its deferral, attempt, routing,
and telemetry behavior. Admission and handler destruction are protected
independently, including forced cancellation; application panic history keeps
the actual Tokio task identity.

Qualification also reproduced an inherited SQLite read-to-write contention
failure on unmodified main. The approved fix reserves writer access with
BEGIN IMMEDIATE for the five read-before-write transaction paths. Pool return
rolls back transactions left by cancellation between SQLx's custom-BEGIN
acknowledgement and creation of its transaction guard. Rollback errors use
SQLx's existing reporting and connection-discard path. Existing lock timeouts,
attempt accounting, retry classification, and stored formats are unchanged.
Two event fixtures now establish their required ordering instead of racing
dispatch; their behavioral assertions and deadlines remain intact.

flowchart TD
    Stop["Accept stop: prevent new work"] --> Grace["Allow admitted work to finish during grace"]
    Grace -->|Work finishes| Cleanup["Wait for tracked native and registered foreign cleanup"]
    Grace -->|Grace expires| Force["Abort remaining async tasks"]
    Force --> Cleanup
    Cleanup --> Result["Return Drained, Forced, or a cleanup error"]
    Cleanup -.-> Pending["Blocked work can keep cleanup pending indefinitely"]

Native grace limits the wait before force, not the total cleanup wait.
Language SDKs retain cleanup and separately limit their callers' waits.
A timeout does not permit callback release.
Independent providers, Clients, and provider token-refresh tasks keep their own lifetimes.

Review focus

Review lifecycle ownership in src/runtime/mod.rs, invocation_scope.rs, and
task_safety.rs; admission/destruction in dispatchers/worker.rs; early fault
reporting in execution.rs; and SQLite transaction acquisition/pool reuse.

Protected internal and cleanup faults request zero-grace shutdown of the
affected Runtime. This can interrupt unrelated work and cause retries or
repeated effects. Ordinary application errors and handler-poll/admission panics
remain operation-local. The catch boundary determines fault scope; a destructor
panic is not always runtime-wide. Admission-guard destruction faults are cleanup
faults, not admission rejections.

Stop status is retained and can escalate from Requested to sticky Faulted.
NotRequested is not a readiness check.
Neither status nor observer closure proves cleanup or permits resource release.

Finalization admits a batch; it is not another replay stage. Normal replay must
reproduce admitted durable operations before history matching reaches them.
A detected pre-snapshot finalization failure rejects the attempt's new batch.
Later faults preserve a valid decision, but earlier force can cancel an
uncommitted acknowledgement. Committed history stays committed. Shutdown adds
no transaction, compensation, or exactly-once guarantee.

The three existing review corrections remain in place: faults precede held
provider reads, observation retains only metadata, and two-turn controls
distinguish normal replay from unsupported finalizer-only scheduling.

The lifecycle guide
documents these contracts with examples and ownership diagrams.
Two older progress records remain clearly labeled historical; their rationale is preserved.

SDK follow-up

Passive observation is now implemented and pushed on the coordinated branches:
microsoft/duroxide-net#21, microsoft/duroxide-node#23, and microsoft/duroxide-python#25.
Default .NET hosting provides diagnostic-only monitoring of the actual selected
lifecycle owner. Unsupported custom owners receive an explicit warning; capable
owners expose the optional observation capability without forcing activation.

Observer cancellation, closure, and source completion remain separate from
execution cleanup. No implementation automatically stops a host, terminates a
process, or restarts a worker. These are branch implementations, not published packages.

Current local qualification

Qualified source bytes match the reconciled commits. Results are Windows x64:

Each core feature mode covers all 61 suites and retains one existing ignored
historical fixture recorder. The unchanged exhaustive cancellation harness
enumerates 587,844 cases, individually asserts 293,922 new-policy cases, and
retains its legacy digest. Only that harness uses test opt-level 2, with debug
assertions and the original 180-second guard; other suites remain unoptimized.
The earlier unoptimized timeout is preserved, not counted as a pass.

A separate proof links a real non-test, hook-free core: seven public lifecycle
and three foreign-retirement metrics cases pass. Cargo's ordinary test builds
are not presented as equivalent hook-free evidence.

The eight SDK cells total 711 passes and 46 explicit hook-only skips.
.NET native DLLs use release builds; Node uses NAPI dev builds; Python uses
dev ABI3 wheels. Actual loaded native paths/hashes and sole local-core/provider
graphs are checked. .NET production compiles out its private-hook cases.
Earlier native assets are not substituted for these builds.

Formatting and all-target/all-feature Clippy pass. Both Rustdoc builds succeed
with seven link warnings in unchanged source files. Each doctest mode passes
46 cases with 74 existing ignored examples.

The October 28 qualification record
contains earlier bounded release-profile performance measurements.
Those are historical, not new measurements of this integration, and do not
establish end-to-end performance or zero overhead.

Remaining gates

Maintainer re-review/approval, merge, and compatible core publication precede
final SDK registry pins and locks. Final registry-only, hosted, release-profile,
supported-platform, and installed-package qualification remain open.
Node's imported provider update does not qualify a shared production PostgreSQL migration.
Current-head hosted CI is a separate gate; these local results do not claim it has passed.
The Node run at the reconciled head
currently fails to compile against registry duroxide 0.1.32: that published
source lacks the unreleased lifecycle/observer APIs, including
RuntimeStopObserver, RuntimeStopStatus, and Runtime::prepare.
This is the compatible-core-publication gate above, not evidence that the
local-source integration has registry or platform qualification.

No release version, tag, package publication, or publishing dispatch is requested.
Local workflow artifacts and binaries are excluded.

Generated with PAW.